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Chapter 1 
SUMMARY 



"Educational research in the United States is going through a period 
of agitation, ferment, and perhaps even crisis." Although Hendrik 
Gideonse wrote these words in 1969, they could not be more relevant in 
1972. The massive, almost ten-fold increase in appropriations by USOE 
for "research and training" between the early and late 1960's was a major 
funding discontinuity which saw the establishment of many new education 
R&D institutions and unprecedented federal commitment to "disciplined 
inquiry" that brought the terms educational development , diffusion , and 
evaluation into general usage. By the late 1960‘s there were heady 
projections suggesting a most likely five-fold (and as much as a seven- 
fold) increase in the numbers of educational research, development and 
diffusion personnel (Clark and Hopkins, 1969). Nearly 100 research train- 
ing programs were funded by USOE to provide an adequate supply of trained 
manpower, but few of these training programs were initially prepared to 
cope with the new areas of D,D&E, The directors of the new federally 
funded projects and programs soon expressed concern over the lack of 
appropriately trained talent. Interest and concern were so great that 
Gideonse (1969, p.v.) noted that in 1968 and 1969 alone "no less than 10 
studies have been or are being conducted on educational research and 
development." By 1972 there are several times as many studies. 

Yet all has not been well. The three-year period of virtually level 
funding for educational R&D "whether measured by United States Office of 
Education, National Science Foundation, or other Federal agencies' appro- 
priations" as Gideonse noted in 1969, has now extended into its sixth year 
and is set in a context of decelerating expenditure and significant read- 
justments throughout the entire American R&D Community (Michael March, 

1970; NSF, 1970). The graduate schools throughout the country, now geared 
to produce needed scientific personnel, are producing an ample supply for 
a retrenching scientific labor market. Current unemployment and projected 
oversupply of scientists and engineers is now a major problem for the 
physical sciences and a matter of concern for the behavioral sciences 
(see Chapter 3. pp. 3.27-3.29). 

At the same time that the educational R&D Community has been con- 
fronted with several years of limited or no real growth in federal funds, 
it has been reproached for its lack of productivity and discernible impact 
on educational practice in both the Congress and the Executive Branch, 
Legislators and acfrninistrative spokesmen have made it clear that extensive 
increases in funding may come only when a markedly better case for 
educational R&D investment can be made and practical evidence of impact 
on improvement of educational practices is submitted. 

Some educational researchers have placed their hopes In a National 
Institute for Education (projecting a vision of accompl ishment .simi 1 ar to 
that of the National Institutes of Health), that could "put it all to- 
gether" in making a case for increased educational R&D funding. The size 
and character of the National Institute for Education and the direction it 
may give to educational R&D, is still conjectural, but it is apparent that 
the United States Office of Education has increasingly committed its scarce 
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discretionary R&D dollars to problem solving, mission oriented efforts 
such as Right-to-read, Career Education, Experimental Schools and a pro- 
posed Renewal Thrust' as well as a broad array of development and dissemi- 
nation efforts to aid the handicapped). 

All of these efforts are relatively massive and complex, indeed almost 
overly ambitious in their goals and objectives, given the short: time lines, 
limited funds, fragile knowledge and technology base, and possible lack of 
sufficiently trained and experienced personnel. 

This then is the context for educational RDD&E in the early 1970's. 

Yet surprisingly little is known with assurance regarding either supply 
or demand for trained personnel , despite the many volumes of reports and 
studies. This situation exists because there has never been a nation-wide, 
in-depth, probability sampling survey of supply and demand for educational 
research, development, diffusion and evaluation personnel. The desire to 
rectify this condition led to the conception of the biennial survey, which 
is fully outlined in Chapter 2, 

An extensive preliminary study and analysis (described in Chapter 3) 
coupled with the knowledge that funds for the 1972 survey would be limited, 
led to recommendations for a more modest initial survey focusing on an in- 
depth study of the dem and side of the problem for only three priority pop- 
ulations: (a) the “core 11 of federally funded educational RDD&E performers, 

(b) the RDD&E activities in local educational agencies (LEA 1 s ) , and (c) 
federal monitors and other federal professionals and paraprofessionals en- 
gaged inor administering educational RDD&E. A remarkably comprehensive 
survey of RDD&E in state educational agencies, to be published in 1972, 
should round out information on these RDD&E populations which appear to be 
critical for the articulated execution of USOE priority programs. 

The recommended survey has been del i beratedly designed so that per- 
sonnel and training requirement projections can be made on the basis of 
program funding information, with reasonable accuracy into the short and 
mid-range future. The survey will also lay the groundwork for a more 
comprehensive survey of both supply and demand at a later time. 

The survey recommendations (see Chapters 4, 5 and 6) provide options, 
but the recommended method is basically a mail survey of federally funded 
contractors/grantees and LEA employers and employees, augmented by field 
interviews and/or telephone interviews to provide greater in-depth ex- 
ploration of complex or sensitive topics. 

The sampling frames for federally funded performers are the federal 
agency lists of current contracts and grants, stratified by level of fund- 
ing converted to a twelve-month equivalent basis. The funding information 
provides a measure of size for estimating number of personnel and for se- 
lecting projects as the primary sampling units. 

Employees are subsampled from lists of personnel supplied by the 
sampled employers (or supplied from project budgets and corrected by em- 
ployers). The sampling of employees is self-weighting, with the sampling 
fraction for primary units proportional to funding size and the sampling 
fraction for personnel inversely proportional. The sampling scheme for the 
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Local Educational Agencies is roughly similar except that size of student 
enrollment is used, as the measure of size to estimate numbers of RDD&E 
personnel, and as a basis for stratification. Because of the very low 
incidence of RDD&E activity in school districts below 12,000-student 
enrollment, it is recommended that the population either be truncated at 
this point or that a screening instrument be used to identify districts 
below this size with significant RDD&E activity. 

The questions included in the draft questionnaire for employers and 
employees, and the recommended analysis, are fully detailed in Chapter 7, 
Questions addressed to employers relate to project (or LEA Unit) identi- 
fication, project content and objectives or unit functions, level and 
sources of funding, major areas of work effort, personnel composition, 
recruitment, selection, attrition, anticipated hires, and training methods. 
Questions addressed to employees include information regarding age, sex, 
race, length of time on job, salary, supervisory responsibilities, job 
satisfaction and advancement possibilities, level of education, courses 
taken and major field for highest degree, types and amount of previous 
work experience and its perceived relevance to the current job, current 
involvement and interest in further training for 25 RDD&E activities, and 
value placed on various approaches to training and credit for training. 

Data for employees can be linked to data for employers so that relationships 
among more than 250 variables may be examined. 

Chapter 4 presents information on the design of the study, including 
discussion of the overall design, sample selection and instruments. Data 
collection is treated in Chapter 5 and creation and use of an Educational 
RDD&E Data Bank is described in Chapter 6. 

Chapter 7 contains an outline of the recommended survey report, with 
extensive discussion of the mail survey data and its analysis, and illustra- 
tions of potentially important or informative analyses which may help to 
define the characteristics, personnel structure, activity, needs and re- 
quirements of educational RDD&E employers and employees in these per- 
former populations. 



O 

ERIC 



8 



Chapter 2 



PURPOSE AND STATEMENT OF PROBLEM 



Purpose and Objectives 

The purpose of this study was to provide the Research Training Branch 
(RTB), Division of Research and Development Resources (DRDR), of the US 
Office of Education, with the design for a biennial Educational RDD&E Per- 
sonnel Supply and Demand Survey. The survey is needed: 

1. To assess the numbers and types of people currently employed in 
Educational RDD&E- and to estimate the future demand for employees in 
educational RDD&E activities (Demand Survey). 

2. To Assess the numbers and types of people with needed skills 
presently available for educational RDD&E employment and to estimate the 
future supply of available individuals for educational RDD&E employment 
(Supply Survey). 

3. To integrate the information from the supply and demand surveys 
to determine present and future Educational RDD&E training and recruit- 
ment needs. 

Statement of the Problem 



Although several studies relating to educational research, develop- 
ment, diffusion, and evaluation (RDD&E) personnel training issues have 
been undertaken in the past few years (AERA series, 1 971 j Oregon Studies, 
1970-71; Hopkins and Clark, 1969; Hood and Banathy, 1970; Fleury, Cappelluzzo 
& Wolf, 1970; Stufflebeam, 1970) each has been significantly flawed in one or 
more respects and, taken together, they provide at best a patchwork basis for 
establishing and justifying Office of Education programs involving the 
use or training of education personnel. Accurate information does not 
exist concerning such items as the numbers employed, types of employment, 
educational backgrounds, relevant experience, skill shortages, perceived 
adequacy of training programs, etc. The information that is available 
is either out-of-date, biased, confined to specific topics, regional, 
unvalidated, based on too small or inadequate samples, or otherwise of 



•I _ 

R&D is conventionally used as an abbreviation for research and de- 
velopment, This report will employ the convention "RDD&E" to refer to the 
entire set, or some unspecified subset, of research, development, diffu- 
sion, and evaluation. In most contexts the reference will be to educa- 
tional RDD&E. Various authors have used other abbreviations: RDD, RDDDE , 

RDDE, R-D-D-E, etc. These forms will be employed only when needed to 
appropriately reflect an author's designation, RDD or E will be employed 
where emphasis on the four separate activities is required. 



limited value. A "base-line" study is needed — national in scope, tech- 
nically adequate in design, and addressing itself to priority Information 
requirements. 



Survey Design Considerations 

The objectives of the design phase of this study were to select and 
develop instruments, data gathering and processing procedures, analytic 
techniques, and planning information for the later implementation of the 
proposed survey. 

Research Training Branch Questions 

The following are examples of types of questions for which RTB per- 
sonnel seek answers: 

Demand Survey . 

1. How many people are presently employed in educational RDD&E 
activities in the United States? 

a. In what types of activities (RDD or E) are they employed? 

b. At what professional levels (paraprofessional or profession- 
al) are they employed? 

c. What annual salaries do they receive? 

d. How many are employed full time; part time? 

e. What are the proportions for: age, sex, race, geographic 

distribution? 

f. In what types of institutions are they employed? 

g. How have they prepared for the jobs? 

h. How long have they worked in the same job activity (RDD or E), 
level, and institution? 

i. To what extent do they value their work compared to Other jobs 
for which they are qualified? 

j. What future do they see for advancement? 

2. How many additional people will be employed in educational RDD&E 
In 1975? 

How much of this additional employment will result from normal 
turnover (without growth) due to personnel leaving educational 
RDD&E or to promotions? 

Given a non-growth situation, how might changes among needs 
for RDD&E activities be reflected in differing demands for 
job skills? 

How will employment be affected by fluctuation in investments 
in educational RDD or E: 

(1) assuming stability in relationship of RDD&E activities? 

(2) if the relationship of RDD or E activities changes? 

Describe the fluctuations in demand in terms of differing de- 
mand for job skills as a function of fluctuations in educa- 
tional priorities or program funding levels. 
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a. 

b. 

c. 

d. 
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1. How many people are presently available for Educational RDD&E 
employment in the United States? 



a. For what types of ROD or E activities have they received for- 
mal or informal training? 

b. For which professional (paraprofessi onal ) levels have they 
been trained? 

c. What educational background, experience or competencies do 
they possess? 

d. Would they be available for full- or part-time work? 

e. What is the distribution for age, sex, race and geographic 
regi on? 

f. In what types of institutions have the graduates received 
training (preservice, inservice)? 

g. Who has paid for training (individuals themselves, insti- 
tutions directly, federal or state government through in- 
stitutions )? 

h. In what types of institutions are they seeking employment? 

i. How many qualified people trained or experienced in areas 
related to educational RDD&E would be available for employ- 
ment without additional specialized training? 

j. How many people trained or experienced in areas related to 
educational RDD&E would be available with minimal (less than 
six months full time) additional specialized training? 

k. How many people already employed in educational RDD&E could 
be employed at advanced levels (or in related RDD&E jobs in 
more critical areas) if they received additional specialized 
trai ni ng? 

2. How many additional people will be trained in educational RDD&E 
by 1975? 

a. How much of this additional supply of trained RDD&E person- 
nel will result from existing programs, funded at present 
levels and focusing on developing the same skills as in the 
past? 

b. How much of this additional supply of trained personnel will 

result from existing programs, funded at present levels but 

focusing on different RDD or E skills than in the past? 

c. How much of this additional supply will result from new or 

expanded programs developing what kinds of skills? 

Analysis of Supply and Demand , 

1, What are the present educational RDD&E personnel recruitment and 
training needs based on the results of the demand and supply 
surveys? 



a. What types of training and recruitment activities are needed? 

b. For which professional ( paraprofessi onal ) levels should 
training and recruitment activities be developed? 
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c . What should be the focus of training and recruitment with 
respect to demographic characteristics (age, sex, race, geo- 
graphi c regi on) ? 

d. Should training and recruitment procedures be directed to- 
ward the needs of specific types of institutions? 

e. What types of training programs (inservice Q-r preservice) 
are recommended for different RDD&E activities and profes- 
sional levels? 

f. What funding sources and amounts are needed to support RDD&E 
traini ng? 

g. How can qualified people who have training or experience In 
educational RDD&E be recruited for relevant positions? 

h. How can people with training and experience in areas re- 
lated to educational RDD&E be given minimal (less than six 
months) additional specialized training? 

1. How can people al ready employed in educational RDD&E be 
trained to perform at higher levels or in priority areas? 

2. What are the projected RDD&E personnel recruitment and training 

needs for 1975 based on projections from 1972 demand and supply 

information? 

a. What training and recruitment policies should be followed 
if existing supply and demand rates are static until 1975? 

b. What training and recruitment policies should be followed 
if the total supply and demand employment rates are static, 
but mismatched in distribution among RDD or E skills? 

c. What training and recruitment policies will be needed if 
supply and demand levels increase at different rates? 

d. What training and recruitment policies will be needed if 
supply and demand levels increase at different rates and 
focus on different RDD&E skills? 

The above set of questions is neither complete nor precisely stated, 
but it indicates many of the specific questions of interest to USOE Re- 
search Training Branch personnel. 

Design Requirements 

The survey design problem, as initially outlined, is illustrated in 
Figure 1. 

The tasks for the design phase were: 

1. To define and refine the problem as presented by the Research 
Training Branch. 
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Figure 1 



Originally Proposed Biennial Survey of 
Educational RDD&E Personnel & Training 
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2 . To study the feasibility of and need for the proposed survey, 

3. To study and make recommendations regarding (a) instruments; 

(b) data gathering and processing procedures; (c) methods of data 
processing, storage, retrieval and possible updating of a data 
bank; (d) data analysis; and (e) planning of the survey. 

4. To select or develop and pretest instruments to collect informa- 
tion on educational RDD&E supply and demand related to the ques- 
tions described above, 

5. To specify methods for defining, selecting and sampling target 
populations, and for reaching the target samples and accomplish- 
ing follow-ups; and to make time, facility, personnel, and cost 
estimates for these methods. 

6. To specify methods for data processing, creation of a data bank, 
storage, retrieval, and possible updating; and to make cost esti- 
mates for alternatives. 

7. To develop a matrix of all supply and demand variables to aid 
analysis and retrieval; design a flexible system of analysis, 
and specify procedures for analysis; and to make cost estimates 
of alternatives. 

8. To specify procedures for improving instruments, data collection, 
processing and analysis for the biennial survey, 

9. To specify procedures for identifying trends in the biennial 
survey, and to verify or revise three-year projections, 

initial Analysis 

Chapter 3 describes the preliminary analysis and planning work which 
was undertaken related to steps 1 (define and refine the problem) and 2 
(study the feasibility of and need for the proposed survey). The con- 
ceptualization of the problem and the development of a proposed solution 
proved to be much more difficult than this report may suggest. Some of 
the problem areas are discussed below. 

Definitions of populations . Educational research, development, dif- 
fusion and evaluation as separate functional activities have been recog- 
nized for hardly more than a decade and are not well defined.* There 
are questions regarding what basic research, if any, should be called 
educational research. What is the extent of applied educational research? 



2 The Oregon Studies (Teaching Research Division, 1971) presents an 
unusually exhausting discussion of current conceptions of educational RDD&E. 




14 t- 



2,7 



For instance, does it include training research, instructional media re- 
search, etc.? Does collection and analysis of educational statistics 
qualify:, or analysis of demographic, public opinion and cost information 
needed for planning and management of local or state education operations? 
How about an Army-sponsored research study of peer tutoring training 
methods? What is educational development? Would a commercial "educa- 
tional" toy that has been modified based on consumer market tests of pro- 
totypes be considered a development? How about the third revision of 
Professor Smith's course in new math, or a computerized system for re- 
trieval and display of student guidance information? Where does the field 
of educational diffusion begin and end? Are textbook salesmen part of the 
diffusion/dissemination labor pool? How about the instructional materials 
center librarian? Who in education isn't involved in "evaluation"? 

Doesn't every teacher and instructional supervisor engage in educational 
research, development, dissemination and evaluation? And how do we de- 
cide what is educational and what is not educational RDD&E? Is it educa- 
cational RDD&E when business, industrial, military, or other organizations 
employ the same general procedures as those employed by educators for 
producing, evaluating or communicating reliable knowledge or practice 
aimed at training or providing people with more adequate methods of 
coping with the demands of their jobs or lives? 

This is a sampling of questions to illustrate the problem of de- 
fining appropriate populations of: 

Educational RDD&E performers (demand) 

Personnel with appropriate competence (current supply) 

Personnel with related competence (short-term trainable supply) 

Training agency programs and capacity (future supply). 

Some of the literature reviewed in Chapter 3 deals specifically with 
the definitional problem. 

Survey Methods . The selection of a survey method or mix of methods 
is generally contingent on many factors such as the size and distribution 
of the population, the size and type of sample, the scope and complexity 
of the information required, the character of the informant population, 
the desired reliability and validity of the information, the desired de- 
gree of accuracy and precision of population estimates from sample data, 
availability of previous information or experience in conduct of and re- 
sults from previous surveys of the same or related populations regarding 
similar subject matter, and of course the available time, financial and 
technical resources. 

Initially, information concerning most of these factors could be 
described as "fuzzy," There were a number of previous surveys of educa- 
tional research. Indeed Barger, Cuba, and Okorodudu (1965) had created 
a National Register of Educational Researchers . There were several stud- 
ies of USOE sponsored research training programs (e.g. Fattu, 1960, 1967 ; 
Lazars feld and Sieber, 1964; Krathwohl , 1965; Sieber and Lazarsfeld, 1966; 
Millikan, 1967; Sieber, 1968; Fleury, 1968). But there were few, if any 
published surveys of educational development, dissemination or evaluation 
that could be considered both comprehensive and technically adequate. 





2.8 



The kinds of technical information that permit optimal design were 
generally not available. The estimates regarding sizes of populations 
and subpopulations were gross; "related" RDD&E populations were best de- 
scribed as "abstractions"; variance data on types of items relating to 
questions such as those previously listed were generally non-existent. 
Discussions regarding the level of accuracy or precision .actually re- 
quired for specific items of information led to the conclusion that the 
major design constraint would be available funds. It was more a question 
of what could be learned about many different types of information with 
a quite limited budget than of what it would cost to obtain a well speci- 
fied but limited set of survey requirements. 

We assumed that some type of mail survey would be employed, and pro- 
ceeded to initiate work on selecting items, development and pretesting of 
questionnaires suitable for mail survey of RDD&E employers and employees. 
But we also initiated investigations of the feasibility and costs of field 
interview and telephone surveys. A major continuing problem was to "scope- 
down" the objectives of the survey and find appropriate "trade-offs" among 
the design requi rements and possible solutions, in the light of emerging 
design information and of discussions with USOE personnel regarding pri- 
orities and probable available funding for the survey. 

Available sampling frames . Assuming that one has achieved well de- 
f i ned (and probably quite arbitrary) definitions of the educational RDD&E 
supply and demand populations suggested in Figure 1, there may be major 
problems in acquiring defensible lists of the members of these popula- 
tions or alternately in prescribing feasible methods for producing the 
required lists. For instance, what registers, directories, membership 
lists would one use, if any, to locate persons "trained or experienced 
in areas rel ated to educational RDD&E" who might, with less than six 
months' full-time training, fill (specified) RDD&E requirements for trained 
personnel? Or, how does one develop lists of non-profit agencies perform- 
ing educational research or educational evaluation work? Or, given lists 
of U.5. colleges and universities, how does one proceed to construct and 
validate a list (or proceed in some other way to identify) departments, 
outside the college of education, which may contain programs or courses 
related to educational RDD&E training requi rements? 

The problems of developing sampling frames are different for RDD&E - 
performing agencies, personnel, and training agencies. If the performer 
group is restricted to federally funded activities the problem may be dif- 
ficult but is at least tractable (see Gideonse, 1969; Clark and Hopkins, 
1969). If educational agencies (regardless of funding source) are in- 
cluded, fortunately there are numerous directories and in some cases use- 
ful statistics (e.g,, USOE National Center for Educational Statistics, NEA 
Research Division, NSF Office of Economic Manpower and Manpower Studies. 
Locating foundation-supported performers is a more tedious proposition, 
although the Science Information Exchange Is a good place to start, fol- 
lowed by query of foundations supporting educational activity. Locating 
relevant educational RDD&E in business and industry, if it is not feder- 
ally supported, appears to be most difficult. The National Science Founda- 
tion does prepare reports on R&D in industry, based on U.$. Census data; 
but neither the fields of basic research, nor the applied R&D product 
groups, nor the industrial classification, is relevant. Apparently the 
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most productive approach may be to attempt to identify industrial per- 
formers through their associations (e.g. Association of American Publishers, 
Information Industry Association, National Audio-Visual Association, Asso- 
ciation for Educational Communications and Technology, Educational Media 
Council, American Society for Training and Development). 

One could attempt to locate performing agencies through association 
membership of their employees. In fact, in the case of the American Psy- 
chological Association Pi rectory this is easily done since there is a 
Geographical and Institutional Directory. 

There are two main approaches to identifying educational RDD&E per- 
sonnel: (a) locating them through their employers, or (b) locating them 

through their associations. Sampling survey methods for selecting educa- 
tional personnel through their employers have been used in studies of 
related RDD&E subject matter (Hood and Hayes, 1967; Chorness, Rittenhouse 
and Heald, 1968), When adequate employer samples can be generated and em- 
ployer cooperation can be enlisted, this approach can be quite efficient. 

Unfortunately the AERA Regi ster (1965) is out of date. We explored 
a number of possibilities including use of the NSF National Regi s ter of 
Scientific and Technical Perso nn el , and use of directories of professional 
associations. Tne results are described in the next chapter. Briefly 
stated, we concluded that an approach through associations would probably 
be attractive only if a "census" of educational RDD&E personnel were needed 
in order to create a new directory or a talent search data bank. It is 
unlikely, given the present situation, that this approach would yield any 
better population estimates (than sampling employers and subsampling their 
employees); and the approach would undoubtedly be more costly or technic- 
ally difficult. 

There are other approaches than the two described above. One could, 
for instance, build lists of personnel based on their publications by using 
ERIC computer tapes. Research in Education , Education Index , Psychological 
Abstracts , etc. The problem with this approach is that it would be highly 
biased, i.e., grossly over-represent'' ng senior researchers located in 
academic institutions and under-representi ng professionals (and certainly 
paraprofessionals) engaged in DD or E in any setting. 

When we turn to the question of locating personnel in related RDD&E 
areas, there are major definitional and technical problems. Reports on 
personnel and training produced by the National Science Foundation, the 
Office of Scientific Personnel of the National Research Council, and by 
the National Institutes of Health provide pertinent background information. 
Chapter 3 summarizes the more pertinent findings regarding numbers of 
personnel from various scientific disciplines contributing to federally 
funded educational RDD&E programs, field switching rates, etc. Unfor- 
tunately detailed information regarding specific skills possessed or 
training needed is not adequate even for those already employed in edu- 
cational RDD&E. Until more detailed information regarding currently per- 
forming personnel is developed, the problem of defining useful questions 
for "related" personnel to answer seems to be an even larger problem than 
constructing sampling frames. 
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The several studies of education research training programs cited 
above provide guidance for the sampling of these programs. The most re- 
cent study (Byers, 1971) is in fact sufficiently recent and comprehensive 
so that we question the need to repeat such an effort in 1972, What is 
most missing from all the training program studies is information on pro- 
grams and courses that may be available in "related" areas. This is simi- 
lar in nature to the problem discussed above regarding "related" personnel 

Sampling method s. One of the few early design constraints we set 
constraint was that the survey results should yield reasonably accurate, 
if not highly precise, estimates of important characteristics for the de- 
fined populations. We knew that several surveys and studies of segments 
of the educational RDD&E population had been recently completed or were in 
progress; but, with possibly one or two exceptions, 3 none were of suffi- 
cient scope or of technically adequate design to afford much in the way of 
useful estimation of required population characteristics . 

Hence, some type of random sampling seemed essential. The practical 
task, once populations and frames had been defined, would be to examine 
alternatives for producing a reasonably simple, practical and efficient 
sampling design that would also have desirable statistical and data analy- 
sis character i st i cs . This did not appear a difficult problem and we had 
expert consultant assistance available. 

Data collection and processing . Compared to other areas, problems in 
this area seemed to be relatively simple, and amenable to standard proced- 
ures. Possibly the greatest problems would be in securing access to and 
cooperation of knowledgable personnel in performing and training agencies. 
Another problem appeared to be in defining questions with sufficient pre- 
cision so as to elicit appropriate and interpretable responses . * 

Data p rocessing, storage and retrieval . Our consultants were able 
to identify a number of available file creation, management and analysis 
programs which seemed suitable for the general requirements that had been 
outlined. This did not appear to be a major problem area. 

Data analysis and Interpretation . For the most part, the data to be 
analyzed seemed to lend itself to relatively standard methods of analysis 
and display. One problem for analysis would be specification of priori- 
ties for areas of interest, since the number of combinations and parti- 
tions of data that could be accomplished would be enormous. A more 
serious problem appeared to lie in the selection of suitable methods for 
projecting supply and demand. And, as we struggled with questions of 
definitions of popul ati ons , problems of reaching appropriate .respondents 




3 Bri ckel 1 (1970) conducted a survey of RDD&E in all 50 of the state 
education agencies. Byers (1971) made an extensive survey of Educational 
RDD&E training programs in graduate schools of education. In both cases 
the surveys approximate a census of the defined populations. 

^Early experience with pretests of mail survey questionnaires made 
us increasingly aware of problems relating to enlisting cooperation, ob- 
taining "sensitive" information, securing adequate response rates, and 
suitably framing questions and response alternatives. 
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and securing their cooperation, and of framing questions, we became more 
concerned with the need to sub-sample to secure in-depth interview data, 
and to follow up on samples of non- respondents in order to guard against 
misinterpretation of the analyses. 

Given the above list of initially perceived problem areas , we searched 
the literature, met with consultants, made visits to a number of federal 
agencies and professional associations, studied published and unpublished 
data and reports and conferred with USOE personnel. These activities and 
findings are reported in Chapter 3. The conclusions described in Chapter 3 
indicated that there was insufficient information and certainly nowhere 
near the financial resources to make a technically adequate, cost-effect- 
ive, comprehensive approach to the proposed survey as initially outlined. 

Consequently, a markedly more modest initial effort, surveying only 
two priority subpopulations of education RDD&E performers (federal grantee/ 
contractors and local educational agencies) and their employees was recom- 
mended. The recommended design, which is fully described in Chapters 4 
through 7, focuses most heavily on the personnel and training demand side of 
the problem and will provide accurate estimation of currently perceived 
requirements and a reasonable basis for projecting personnel and training 
demands in these two priority areas. The survey has been designed so that 
subsequent surveys, designed on the basis of findings and experience with 
this initial survey, should provide more effective and less expensive 
survey information dealing with the range of educational RDD&E personnel 
and training issues confronted by the RTB questions listed above. 
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